1 Image Distortion for Gun Sighting and Other Applications 

2 

3 Field of the Invention: 

4 [0001] The present invention relates to cameras and image display systems, 

5 and more particularly to such systems which provide images that distort reality 

6 for particular purposes. 
7 

8 Background of the Invention: 

9 [0002] Lead gun sights that compensate for target motion are well known. In 

10 general such gun sights provide a targeting cross hair at a position removed from 

11 directly in front of the gun barrel. For example U.S. patent 5,127,165 describes 
|£j 12 an electronic system which generates a cross hair in a gun sight at a location 

□ 13 which takes into account motion. U.S. patent 5,067,244 provides a list of prior art 

;Vj 14 patents directed to various aspects of "lead gun sights". 

?! 15 

Lil 16 [0003] Weapon control systems have been developed which calculate and 

17 take into account the ballistic characteristics of projectiles when aiming various 

[l| 18 weapons in response to signals such as radar signals. For example see issued 

Ij 19 US patents 3,845,276 and 4,146,780. 

I 20 

21 [0004] The present invention can accomplish the same general objective as 

22 the above described systems; however, the objective is accomplished in an 

23 entirely different manner. Furthermore, the present invention can be used for 

24 other purposes. The present invention utilizes imaging technology in 

25 combination with computer calculations. The technology for capturing and 

26 displaying panoramic images is well developed. For example see U.S. patent 

27 6,337,683. Such technology can capture a plurality of images, seam the images 

28 into a panorama and display a view window into the panorama on a computer 

29 monitor. 
30 
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1 [0005] The present invention utilizes imaging technology and the technology 

2 that can predict the trajectory of a flying object in a new combination. With the 

3 present invention an operator is presented with a panoramic wide view image 

4 that provides perspective to any targets reachable by a weapon and at the same 

5 time conveys appropriate targeting information. The purpose of the present 

6 invention is to provide a wide angle image which is predictively distorted so that 

7 an operator can easily visualize targets in an entire theater of operations and so 

8 that an operator can easily determine which targets are in the range of his 

9 weapon. The present invention also has applications beyond providing an image 
10 to aid in aiming weapons. 

M* 11 

CJ 

Q 12 

W 13 Summary of the Present Invention: 

111 14 [0006] The present invention provides an operator with a predictively distorted 

1 1\ 1 5 display of a theater of operations. An image of the theater is acquired with a 

16 conventional camera and then the acquired image is distorted to take into 

17 account environmental factors such as air speed, ground speed, wind speed, 

18 height, exact distance to target, etc. For example in a simple embodiment of the 
g 19 present invention can be used where a platform such as an airplane is moving 

20 over a geographic feature and objects are being dropped from the platform. With 

21 the present invention, a geographic feature that is actually directly under the 

22 platform is made to appear on a display as if it is behind the platform. The 

23 reason for this is that if an object is dropped at a particular instant, it can only 

24 impact at positions that at that moment are ahead of the platform. Hence, 

25 positions ahead of the platform are made to appear directly under the platform. 

26 The amount that each pixel in the display is distorted takes into account the both 

27 the speed of the platform, the aerodynamics of any projectile, and other 

28 environmental factors. The invention can be used to provide a display that an 

29 operator would use to aim a weapon at a target. The invention can be used to 

30 predictively display an image of an environment that takes into account any 
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1 known and/or predictable relationships between a moving platform and the 

2 environment. 
3 

4 [0007] The preferred embodiment of the invention includes a camera (or other 

5 image capturing device such as radar, sonar, etc), a computer programmed to 

6 predict the affect of relative motion between the platform and the environment 

7 and a display to show the distorted predicted view of the environment. 
8 

9 Brief description of the drawings: 

1 0 [0008] Figure 1 A and Figure 1 B illustrate the pixels of an image. 

11 Figure 2A and 2B illustrate a moving platform relative to a number of identified 
Q 12 points. 

( :| 1 3 Figure 3 is a system block diagram. 

W 14 Figure 4 is a program flow diagram. 

!l 

N 15 

16 Detailed Description: 

Q 17 [0009] In a first embodiment a digital panoramic image is acquired and 

r|j 18 seamed in a conventional manner. For example a panoramic image can be 

|;Jj 19 acquired and seamed as described in U.S. patents 6,337,683 and 6,323,858 

if 20 and in co-pending application 09/602,290, filed 6/23/00 entitled Interactive 

21 Image Seamer for Panoramic Images" the content of which is incorporated 

22 herein by reference. 
23 

24 [00010] A digital image consists of an array of pixels. Figures 1 A and 1 B 

25 illustrate, in greatly exaggerated fashion, a few pixels from an image. An actual 

26 image would contain many thousands of pixels; however, for convenience of 

27 illustration, only a few of the pixels are illustrated in Figures 1 A and 1 B. Often 

28 with a panoramic image, only a selected view window into the panorama is 

29 displayed. The pixels illustrated in Figures 1A and 1B can be taken to represent 

30 some of the pixels in a view window or a subset of the pixels in an entire 

31 panorama. 
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1 

2 [00011] The pixels shown will be referred to by their coordinates. For example, 

3 the pixel at the top row on the left will be referred to as pixel 11 , the fist pixel in 

4 the second row will be referred to as pixel 21 , and the second pixel in the second 

5 row will be referred to as pixel 22. 
6 

7 [0001 2] A system diagram of a preferred embodiment of the present invention 

8 is shown in Figure 3. The system includes a panoramic camera 301 on a moving 

9 platform such as an airplane (the platform is not shown in the Figure). The 
10 camera 301 records an image. The image recorded by the camera is 

U 1 1 "predictively distorted" in a manner that will be explained later. The predictively 

H 12 distorted image is presented to an operator on a display 308 to help the operator 

OB 1 3 take some action such as aiming a weapon 307 or dropping a bomb. 

W 

if 14 

^ 1 5 [0001 3] With the present invention, the value of each pixel in the perspectively 

» 16 distorted display either corresponds to a selected pixel (called the source pixel) in 

ill 17 the recorded image or it is generated or modified to provide a calculated artifact 

f5| 1 8 (such as the fact that a certain area is out of range). It is important to note that 

Q 1 9 the location of the pixel in the perspectively distorted display can be different 

: * 20 from the location of the related source pixel in the recorded image. 
21 

22 [00014] Figure 1 A illustrates some of the pixels in the recorded image and 

23 some of the pixels in the perspectively distorted image that is displayed. The 

24 point of Figure 1A is to illustrate that the value of pixels in the displayed image 

25 can originate from a source pixel in the recorded image; however, the location of 

26 a pixel in the displayed image does not generally coincide with the location of the 

27 corresponding source pixel in the recorded image. 
28 

29 [00015] In the following discussion a pixel will be described as having been 

30 "moved" when the location of the source pixel in the recorded image does not 

31 coincide with the location of the corresponding pixel in the displayed image. The 



1 movement of pixels will be described in terms of vectors. Examples of such 

2 vectors are illustrated by the arrows shown in Figure 1 B. 
3 

4 [00016] In the example shown in Figure 1A the illustrated pixels are moved as 

5 follows where the numbers given are the location index values of the pixels: 
6 



Pixel location in 
source image 


Pixel moved to this 
location in Distorted 
Image 


7,6 


4,6 


7,7 


4,7 


7,8 


3,9 


7,9 


3,10 



7 

8 [0001 7] The above table is merely an example showing how a few pixels are 

9 moved. The above example shows that different pixels are moved by different 

1 0 amounts. Most pixels in the distorted image will have a corresponding source 

1 1 pixel. If there is no source pixel for a particular pixel in the distorted image, 

1 2 interpolation will be used to determine the value of the pixel from the value of 

13 adjacent pixels. 
14 

1 5 [0001 8] The display presented to the operator consists of the pixels in the 

16 panorama (or in the view window) each of which has been moved in accordance 

17 with the vectors applied to that particular pixel. The result is somewhat similar to 

18 what would happen if the pixels were dots on a rubber sheet and the sheet were 

1 9 stretched in a number of different directions. It is however noted that with a 

20 rubber sheet the spacing of the dots on the sheet changes as the sheet is 

21 stretched. However, the pixels in the recorded image and the pixels in the 

22 predictively distorted display have a particular spacing determined by the 

23 characteristics of the display. Where the dots on a sheet do not coincide to the 

24 location of the pixels in the distorted image, interpolation is used. 



1 

2 [0001 9] The distortion which is applied to images with the present invention is 

3 similar to taking an image in a drawing program and morphing the image in a 

4 particular direction. That is one can latch on to a particular point in an image an 

5 pull that point so as to distort the image. With the preset invention such distortion 

6 is done to create a display which shows a theater of operations predictively 

7 distorted to facilitate targeting a weapon such as a gun. 
8 

9 [00020] There can be any number of factors which affect the location of each 

1 0 pixel. In Figure 1 B a number of vectors are shown at the location of each pixels. 

1 1 Each vector represents an environmental factor that affects that pixel. The 

M 12 direction and magnitude of the vector indicates the direction and magnitude of 

9 1 3 the effect. For example one vector can represent how the pixel is moved due to 

Ul 

m 14 air speed, another vector can indicate the affect due to wind velocity at that time, 

I 15 and another factor can represent how a pixel is moved due to the trajectory of a 

y, 1 6 particular projectile. For simplicity of illustration on two vectors are show for each 

17 pixel in Figure 1B. 
m 18 

Pj 1 9 [00021] The invention and its operation will first be described by using a very 

Q 20 simple example. Next the more complicated applications of the invention in a 

ft! 

! * 21 more complicated real world environment will be described. 
22 

23 [00022] A simple application of the invention can be understood from the 

24 following simple example. Consider the following: if while standing in a moving 

25 vehicle one drops an item as the vehicle passes over a particular location, the 

26 item will not hit the particular location due to the motion of the vehicle. With the 

27 present invention, one would observe the environment on a display. The image 

28 on the display would be predictively distorted so that when it appears that the 

29 vehicle is moving over a particular location, the vehicle would in fact not as yet 

30 reached that location. Thus if an item is dropped as one appears (from the 

31 distorted displayed image) to be moving over a particular location, the item would 
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1 in fact hit the location since the display was predictively distorted. This simple 

2 example does not take into account factors such as wind speed and the 

3 aerodynamics of the item. 
4 

5 [00023] Figure 2A illustrates a moving platform 101 which could for example be 

6 an automobile or an aircraft. The stationary environment is illustrated by line 105 

7 which has points 1 to 8. The motion of platform 101 is in the direction of arrow 

8 103. A view 102 which is directly down from platform 101 would focus on point 3 

9 on the line 1 05. Figure 2B illustrates what an operator would observe on a 

1 0 predictively distorted display when the platform 1 01 is at the position indicated in 

1 1 Figure 2A. The operator would see a display that shows the platform over point 
I* 12 5 on line 105 as shown in Figure 2B. Thus, if an operator was looking at the 

!^ 13 points on line 105 when the platform was at the position shown in Figure 2A, the 

pfl 14 operator would see a display which shows the platform at the position shown in 

if j 1 5 Figure 2B. That is, when the platform is at the position shown in Figure 2A, the 

^ 1 6 image on the display would be predictively distorted so that it appears as if the 

17 position is a shown in Figure 2B. 

S 18 

[J 1 9 [00024] The above is a very simple example of the operation of the invention. 

O 20 In the above example, the pixels in the image of the terrain along a line are 

Fin 

1 ** 21 affected by a single vector which moves them backward by an amount 

22 determined by the speed and height of the platform (i.e. the amount is the 

23 distance the platform moves in the time it takes an item to move from the 

24 platform to the ground. Since in this example the item drops straight down, areas 

25 of the distorted display other than the area along the line would be colored or 

26 darkened to shown that only points along the line are available targets. In this 

27 example the pixels are affected by a single vector. In other embodiments the 

28 pixels could be moved in accordance with a number of vectors representing 

29 factors such as wind speed, aerodynamics of the particle, etc. 
30 



1 [00025] Figure 3 is an overall systems diagram of a preferred embodiment of 

2 the invention. The system includes a panoramic camera 301. Camera 301 can 

3 for example be the type of camera shown in U.S. patents 6,337,683 or 

4 6,323,858 However, other embodiments of the invention could alternately use 

5 any one of a variety of other commercially available cameras. 
6 

7 [00026] The system as shown in Figure 3 includes a mechanism 302 for 

8 supplying information concerning environmental factors and data. The data 

9 provided by mechanism 302 can include projectile flight models terrain data. 

1 0 Mechanism 302 can include measurement apparatus that measures 

1 1 environmental factors such as wind speed, air speed, GPS location data, etc. In 
J 1 2 a simple embodiment, mechanism 302 could merely provide speed and height 
3 13 measurement. In more complex systems mechanism 302 could include devices 
! ;t 14 that measures a wide variety of factors such as speed, air temperature, air 

1 1 5 pressure, GPS data, etc. The GPS data which indicates the present position of 

j 16 the camera can be used together with information in the terrain data base to 

!*= 1 7 calculate the distance from the platform to particular geographic features, thereby 

1 1 8 allowing the system to calculate if such geographic features are within target 

J 1 9 range and if so how the image need be distorted to show if the particular feature 

|! 20 can be hit by firing the weapon at a particular time. 
21 

22 [00027] The output of camera 301 and environmental factor measurements 302 

23 are fed into a computer 304. In a simple embodiment, computer 304 could be a 

24 personal computer whereas in a more complex system, computer 304 could be a 

25 remote large mainframe computer that is connected to the remaining elements in 

26 the system by a wireless link. 
27 

28 [00028] The purpose of the entire system shown in Figure 3 is to control the 

29 firing of a weapon 307 that is manually aimed by a control unit 306. A cross hair 

30 308A displayed on display 308 shows the projected impact area of a projectile 



1 fired with the controls set as they are at that moment. As the controls 306 are 

2 manipulated the cross hair 308A moves. 
3 

4 [00029] An operator (not shown in the drawing) manipulates controls 306 while 

5 looking at display 308. The image on display 308 is the type of image illustrated 

6 in Figure 1 . That is, the image displayed is an image of the environment; 

7 however, each pixel has been moved by an amount equal to one or more 

8 vectors. In a very simple embodiment where items are being dropped form a 

9 moving platform, the pixels would merely be moved forward to compensate for 

1 0 the forward speed of the platform. In such an embodiment, the image would not 

1 1 show the ground directly under the platform, instead it would show the ground a 

12 calculated distance in front of the platform. The area shown would coincide with 

1 3 the area where an object dropped from the platform would impact. 
14 

15 [00030] In a more complex embodiment, each pixel would be moved by the sum 

16 of a number of vectors. These additional vectors could for example take into 

1 7 account the speed of a cross wind and the ballistic characteristics of the weapon 

18 being fired. 
19 

20 [00031 ] If for example there were two different types of weapons are on a 

21 platform, the operator of each weapon would see a different distorted image. 

22 Pixels that coincide with areas out of range of the weapons would not even be 

23 displayed on the screen. Thus, the display would illustrate only the area that 

24 could be effectively targeted by a particular weapon. 
25 

26 [00032] Figure 4 is a block diagram of the computer program that produces the 

27 predictively distorted display. The system has two inputs. The first input 401 is 

28 from the camera that captures the image. The second input 402 acquires 

29 various environmental factors that affect each projectile. 
30 



1 [00033] As indicated by block 404, vectors are calculated for the various factors 

2 that affect projectiles filed by weapon 307. This calculation is made using a 

3 mathematical model of the flight path of the projectile which is being filed by 

4 weapon 307. For example, one vector would represent the forward motion of the 

5 platform, one vector would be for the wind velocity. Vectors are calculated for 

6 each pixel position. The vectors indicate the magnitude and direction each 

7 particular pixel must be moved to compensate for the associated factor. The 

8 various vectors that affect each pixel are summed as indicated by block 406. 

9 The sum vector for each pixel is then used to move the particular pixel as 

1 0 indicated by block 406. The distorted image (that is, the moved pixels) is then 

1 1 displayed as indicated by block 408. 
M 12 

N 1 3 [00034] The point of impact is calculated (for the setting of the weapon control 

,;8 14 306) as indicated by block 405. This is done using conventional technology 

fll 1 5 including a model of the weapon 307 and its projectile. The position of the 

Ti 16 crosshair 308A on the display 308 is calculated based upon how the weapon 307 

17 is aimed at the particular moment. 

Q 

m is 

! 5) 1 9 [00035] Areas that are not in the range of weapon 307 are shown with a 

5 20 distinctive color or with cross hatching so that the operator can immediately see 

m 

' * 21 what targets are within range and available. The display thus gives the operator 

22 both a theater wide perspective view and a clear indication of what targets are 

23 available at that particular time. 
24 

25 [00036] The camera can also include a program that detects motion of objects. 

26 For example the fact that a vehicle is moving on the ground can be determined 

27 by comparing two images taken at different times. Such motion detection 

28 technology is known. Where a vehicle or object is moving, this fact can be 

29 illustrated on the predictively distorted display by showing a trail or smear behind 

30 that object to illustrate the motion. 
31 
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1 [00037] While preferred embodiments of the invention have been shown and 

2 described, it will be understood by those skilled in the art that various changes in 

3 form and detail can be made without departing form the spirit and scope of the 

4 invention. The applicant's invention is limited only by the appended claims. 
5 

6 [00038] I claim: 

7 

8 

9 

10 



rii 
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